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The use of Likert items on questionnaires is very popular in sociological and 
psychological measurement In the typical analysis of data from such a questionnaire, 
frequencies of endorsements to points on the scale are tabulated and percentages are 
computed separately for each item and point of the scale. These descriptive statistics are 
inconvenient when we try to compare and interpret subjects' responses as measures en the 
construct we are assessing. Comparisons among items become more cumbersome as the 
number of items or the number of points on the scale is increased. The points on the scale 
are frequently assumed to constitute an equal interval scale and subjects' scores are treated as 
constituting a continuous variable. This assumption is hard to justify if the frequency 
distribution of scores is highly skewed. Item response theory (TAT) models have provided a 
solution to problems such as these in other contexts. 

Item response theory models applicable to scales made up of dichotomously-scored items 
measuring one proficiency dimension have been developed and are now in widespread use. 
The distinct feature of IRT models as compared to other statistical models for categorical 
variables, such as logistic and log-linear models, is the inclusion of latent traits as variables. 
The model that is the subject of this paper is based on two extensions of the basic IRT 
model. Models that can incorporate polytomously-scored items have been proposed and used 
by several researchers (Andrich, 1978, 1982, 1988; Bock, 1972; Masters, 1982; Muraki, 
1990, 1992a; Samejima, 1969, 1972). Bock and Aitkin (1981) extended IRT models for 
dichotomously-scored items to the multidimensional case (several proficiency dimensions) 
and developed an EM algorithm (Dempster, Laird, & Rubin, 1977) to estimate the 
parameters of the model based on the normal ogive. McKinley & Reckase (1983) proposed a 
logistic-distribution-based multidimensional model. In this paper a multidimensional IRT 
model for polytomously-scored items, based on Samejima' s graded response model and 
using the normal ogive, is developed, and an EM algorithm that may be used to estimate the 
parameters of the model, is also discussed. 



Development of the Model 

Bock, Gibbons, and Muraki (1988) assume that the interaction of person i and item j 
results in a response process variable, y s , that is a linear combination of M latent traits. 
Using vector notation in which fi is an M-dimensional vector of latent traits (common factors) 
and a a vector of factor loadings: 

fij " (©lif 6 2i , • • • , 6^) 
ttfj - (a jlf a j2 o^) 

we may write this combination as 



i 3 



'Ml 



+ € 



[1] 



where e a is an unobserved random variable (a unique factor in factor analytic terms) which is 
assumed to be distributed N(0, o- 2 ). Conventionally in factor analysis it is assumed that the 
distribution of £ is N(Q, I) and that y i is distributed with mean 0 and variance 1. Hence the 
unique variance is 



M 



- l - a! a. 



[2] 



Classical factor analysis for continuous variables is based on the assumption that the 
response process is directly observable. In contrast, the factor analytic model for categorical 
variables is based on the assumption that the response process variable, y p , is latent and 
realized into a vector of polytomous item responses for J items, 

M i = (W u , W i2 , . . . , w iiT ) 

according to the psychological mechanism 

y j0 = — 

Yix = 00 

= 1/ 2/ • • • / K) 

where y^ is a threshold parameter associated with category k of a K-category Likert-type 
item, j. The process generates a categorical response of k for person i to item j when 
equals or exceeds the threshold, y sXAt but does not exceed the threshold y^. Assuming a 
normal ogive model the probability of categorical response k by person i to item j given 
his/her M-dimensional latent trait is expressed as 



k | fl J ) = f exp 

Vj.A-t 



\ 2 



dy 



[3] 



Introducing a change of variable, we may re-write the item response model in [3] by defining 
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Then 



and when 



then 



and when 



then 



_ y» - 

c ij — 



• 4 



[4] 



dy dj = OjCfty [5] 



y±j = Yj.jt-i [8] 



**** = ' 1 C 9 3 



Following Bock, Gibbons, and Muraki (1988) we define slope and item-category 
parameters: 



*jm = CIO] 



* °j [in 

(Jc = 1, 2, . . . , 



ERJC 



and define the following functions: 



(k = 1, 2 , .... K-l) 



c»l 



P jk (Q.) = / <i>(fc) dfc - J <J»(t) dfc 
(^c ™ l f 2/ t • ■ / iir ) 



[12] 



p; 0 = o. [i4] 

PilC-l. C«] 

* (t) = 7^ exp ("fl cl63 

then the model in [3] may be written as 
which can be written as 



[18] 



For k= 1 the last term is zero and fo; k=K the first term is the definite integral of a 
probability density function from negative infinity to infinity and therefore equals one. 
Hence we may write these two cases as 

p^m - pi aw 

p JK (Q) = i.o - p; fJM (fl) 
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Equation 17 is referred to as the operating characteristic of the graded response model by 
Samejima (1972). 

Once we obtain the estimates of slope and item-category parameters, we can compute the 
corresponding factor loadings and threshold parameters from relationships derived from [2], 
[10], and [11], as follows: 

* jm = 4** t20] 

and 



where 



$i * i + E4* * 1223 

at"! 



Interpretation of Parameters in Multidimensional IRT Models 

Reckase (1985, 1986) developed "multidimensional" parameters that serve as an aid in 
understanding the nature of items generated according to the multidimensional logistic model 
and Carlson (1987) further elaborated on their meaning. Here we apply similar ideas to 
items generated by multidimensional models for polytomously-scored items and, in 
particular, the normal ogive model in [17] and [18]. 

Following Reckase's (1985, 1986) definition, the multidimensional discrimination of a 
polytomously-scored item can be defined as 



1i - E *i- ' t23] 

JB-l 

and the multidimensional item-category parameter can be defined as 
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Reckase (1985) found the direction of steepest slope of the response surface can expressed in 
terms of angles with the latent trait axes as 



COSto^ = ig! * X Jm [25] 

The slope in the direction specified by angles is at its maximum when the item response 
surface (IRS) of Pjk + (0) crosses the .5 probability hyperplane. 

The item-category parameter of the cumulative probability in [13] is the proficiency 
value at the point of .5 probability of the kth or below kth categorical response where 



Z a ja B u = -b jk [26] 

JD«X 



If we denote a particular dimension as m' (1 < m' £ Mj, we obtain the following equation 
because of the symmetric relation: 

B«- [27] 

Equations [26] and [27] can be solved from the points in the proficiency dimensions locating 
the item-category parameter: 

- 1281 

= P JJt coso iJB / 



From [25] and [28], we obtain the following equations: 
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a 3m * I^OSO)^ 



and 



[30] 



Since the direction cosine is cosfajj for both the multidimensional discrimination ijj and the 
multidimensional difficulty fa, they must reside along the same axis. The slope parameter 
aj. is the mth coordinate of the point where the slope of IRS is the steepest. The value 
indicates the mth coordinate of the point on the line. In other words, it is the projection of 
fa onto the mth 0 dimension. Thus, the parameter, t^, can be referred to the mth 
component of the multidimensional item-category parameter, fa. 

The concepts discussed above can be best explained in the context of two dimensions. 
Figures 1 and 2 show a two-dimensional response surface and contour plot, respectively, for 
Pjt + (0) in [13] with a-parameters of .8 and .6 and b^-parameter of .5. Note that the locus of 
all points for which 0i=O. defines a line in the response surface (Figure 1) which is a 
unidimensional item characteristic curve above the 0 2 axis. Similarly, for 0 2 =O. there is a 
two-parameter unidimensional item characteristic curve over the 6 X axis. The ^ for the 
example item j in the figures is therefore equal to 1.0 and the fa is -0.5. The direction 
cosines of the line of maximum slope in reference to 9 t and 0 2 are .8 and .6, respectively. 
Since these 0 axes are orthogonal, cos 2 Wj, + cos 2 « j2 =l. 



Consider the geometry in Figure 3. The 0, and 0 2 are two orthogonal axes representing 
the two dimensions underlying item j. The linear combination in [12] indicates the 
combination of the two 0s that the item can be considered to be measuring: 



The equation of the line defining the direction of measurement, as is shown in Figure 3, can 
be written as 



Insert Figures 1 and 2 about here 
******************************************** 




9 

ERJC 



7 

s 



The two a parameters are the lengths of two sides (a-i = OC and aj 2 = OA) of a right 
triangle whose hypotenuse (OB) is equal to Reckase's multidimensional discrimination 
parameter, ^. The angle BOC is <a ix and the angle AOB is its orthogonal complement, w^. 
The two r parameters are the lengths of two sides (r lu = OF and T jl2 ■ OD) of a right angle 
whose hypotenuse (OE) is equal to the multidimensional difficulty parameter (0^. The 
multidimensional item response surface of the cumulative probability is a two-dimensional 
surface above the 0 r 0 2 plane (Figure 1), and a slice through that surface along the line of 
measurement of the item is a unidimensional item characteristic curve for an item with 
discrimination parameter equal to ijj and item-category parameter equal to jS* (Carlson, 
1987). 



Insert Figure 3 about here 



By using the multidimensional parameters defined above, the model in [12] can be 
rewritten as 

- r\ 3 (cos&> Jj e i +cos<i> i2 0 2 + . . . +cosu> jl fi lf -$ jk ) 

- ^ (*iA*Ma+« • • + *iA-Pj*> C313 

The unidimensional latent trait 8 * is a composite of M dimensional latent traits 0 m (m=l, 2, 

M). Reckase pointed out that an item, although being scaled in a multidimensional 
context, can be considered to be measuring along a single dimension. That dimension is a 
linear combination of the uncorrelated theta dimensions. The test containing the item may, 
however, be multidimensional if it consists of items that measure along different directions in 
the theta space. Since it is assumed that the distribution of £ is N(Q,I) and the sum of 
squared direction cosines in the orthogonal space is 1., the variance of 0/ is also 1. 

The model can be also expressed as a linear combination of factor loadings and 
thresholds: 

Z jk (Q) = 5j(«jA + «jA + • • • + «iA-Yiic) 

Q 3 
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Unlike the composite latent trait 0/ in [31], the variance of a linear combination of the 
weighted individual 0 m , 0/', in [32] is not 1. The variance of this composite of latent traits is 
called the communality, h, 2 . Since 

Z <fi) - jfl-la,,}) [33] 
Z <fi) ~ 4> [34] 

and 

?3 = l+ti§, [35] 
the communality is expressed by the parameters in [31] as 

hi * [36] 

The multidimensional discrimination and direction cosines are expressed by the 
parameters in the model [32] as 

hi hi 



{i-h 



and 



As seen in [10] and [37], the multidimensional discrimination can be interpreted in the same 
way as the conversion of factor loadings to the slope parameters. Furthermore, the direction 
cosine can be also computed as the ratio of the factor loadings to the square root of the 
communality. 



To summarize, the multidimensional parameters defined for the McKinley-Reckase 
M2PL model (McKinley & Reckase, 1983; Reckase, 1985) can be adopted for the 
multidimensional graded response model in [18] because the cumulative probability of each 
categorical response of kth or below kth, P^+Cfi), is essentially a dichotomous item response 
model and the logistic function is only an approximate form of the normal ogive function. 
The difference is that the multidimensional polytornous item response model yields a set of 
K-l IRSs of Pjk + (fi) rather than one such surface for the dichotomous item response model. 

These IRSs are parallel along the line defined by a set of direction cosines, X^, m=l,2 

M. The probability of the specific middle category k, Pjt(fi) is defined by subtracting P jik . 

from Pjk+Cfi). Since these IRSs of cumulative probabilities are parallel, the 
multidimensional parameters are still meaningful even for specific middle categories. 

Bock, Gibbons, and Muraki (1988) established the relationships between the parameters 
of the factor analysis model and the parameters of the item response model. In this paper, 
we have established the relationships between the factor loadings and multidimensional 
parameters like those in Reckase 's model. The multidimensional parameters can provide 
useful interpretations of parameters of the multidimensional item response models. These 
parameters can be computed directly from the factor loadings. 



Figures 4 through 7 show the IRSs of polytomously-scored three-category item 1 with 
parameters: a„=1.0, a, 2 =1.5, b„=-0.8, and b, 2 =1.2. In Figure 4, the IRS of P,,(fi) is 
plotted. This is the same probability as P u + (0, as given in [19]. In Figure 5, the IRS of 
the second cumulative probability P 12 + (g) is plotted. These two IRSs are parallel to each 
other. The model probability of the second categorical response, P, 2 (g)= Pi 2 + (fi)-Pn + (0, is 
computed and plotted in Figure 6. Finally, the model probability of the third categorical 
response, P i3 (0= 1.0 - P, 2 + (fib), is plotted in Figure 7. If the width of the item-category 
parameters is shortened, the probability of the middle categorical response is uniformly 
decreased. Consequently, the IRS becomes flatter. In Figure 8, the IRS of the middle 
categorical response of item 2 with the same parameter values as item 1 except that b2,=0.8 
is shown. Since the width of the item-category parameters becomes narrower given the same 
slope parameters, the IRS is pushed to downward. We can expect the observed response 
frequencies of the second category to be smaller for item 2 than for item 1. The shape of 
the IRS does not change if we rotate the original IRS. In Figure 9, the IRS of the second 
categorical response, P 32 (0, with slope parameters a 3 ,=1.5 and a 32 =1.0 is plotted. 



******************************************** 

Insert Figures 4, 5, 6, and 7 here 
******************************************** 
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Parameter Estimation 

Let U jjk represent an element in the matrix of the observed response pattern i. 11^= 1 if 
item j is rated by the ith respondent in the kth category of a Likert scale, otherwise U ijfc =0. 
By the principle of local independence (Birnbaum, 1968) the conditional probability of a 
response pattern i, given 6, for K response categories and J items, as denoted by a response 
matrix (UjJ, is the joint probability: 

Pi[(^)ifl] « nn [p ^ (fi>]a ^ [393 

J-i *-i 

For examinees randomly sampled from a population with a multivariate normal distribution 
of the latent trait variable, <£(g), the marginal probability of the observed response pattern i is 

Pil(V jk )] = fp ± l{U jk ) !fi]<i><fi)cft [40] 
fi 

If an examinee responds to J items with K categories, his/her response pattern i can then 
be assigned to one of K J mutually exclusive patterns. Let r x represent the number of 
examinees observed in such a pattern i, and let N be the total number of examinees sampled 
from the population. Then r { is multinominally distributed with parameters N and P{(U^). 
This probability can be interpreted as the likelihood function of the parameters and b^: 




[41] 



Taking the natural logarithm of Equation 41 yields 

lnL = lntf!-£ lnr i !+2^ i lnP i [(^)] C 42 l 

i-i i«i 

Bock and Aitkin (1981) applied the EM algorithm (Dempster et al., 1977) to estimate the 
parameters for each item individually, and then repeated the iteration process over J items 
until the estimates of all the items became stable to the required number of decimal places. 
This is in contrast to the Fisher-scoring procedure of Bock and lieberman (1970). The qth 
cycle of the iterative process can be expressed as 
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[43] 



for each item j. The vector of estimates u represents the model parameters. The orders of 
parameter vector u and gradient vector t are both M+K-3, and the order of information 
matrix V is (M+K-3) x (M+K-3). The information matrix is the negative expectation of the 
matrix of second derivatives. When the number of response categories is K, only K-l item 
category parameters can be specified. In addition, two extreme ends of parameter values 
need to be fixed to estimate slopes. Furthermore, because the covariance between any 
categories that differ by more than two points is simply 0, the partitioned information matrix 
for the category parameter estimation becomes a tridiagonal symmetric matrix (Muraki, 
1990). 

The likelihood equations for and 6^ can be derived from the first derivatives of 
Equation 42 with respect to each parameter, and respectively set to 0. 

With respect to a^, the likelihood in [42] can be differentiated as 

da ja f£ PiUUjJ] J * & da jm [p Jk (fi)]"«*. 



Now let the observed score patterns be indexed by 1=1,2,..., S, where S ^min(N,K J ). If the 
number of examinees with response pattern 1 is denoted by r„ then 



1-1 



The first derivative of the likelihood function in [44] can be approximated by using the 
Gauss-Hermite quadrature, such that 

aim, 

* E E -E E ^^^^^^'^^^ d[p jk (x)]°»* ! 

1-1 £g.l 'Vl ii-l ?l h. fojm [P jk (2£)l U » k 

[46] 

where 
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h = E ~E £ nfi (%(^]^u fi )Au fj )^u fj( ) [47] 



and 



l 2 (20 = nn iPjkWi*"* t 48 ^ 

J«l Jc-i 

In Equation 46, A(Xf) is the weight of the Gauss-Hermite quadrature, and X f is the 
quadrature point (Stroud & Secrest, 1966). The quadrature weight A(Xf) is approximately 
the standard normal probability density at the point Xf for each dimension. Because can 
take only two possible values, 1 and 0, the element of the gradient vector i can be written as 

■ E ~E E E rSa £ ~ 14,3 

where 



s 



r 



r 2 L 2 (20 A {X fl )A (X fi ) ..A (XfJ U ljk 



= £ — v-** [S0] 



and r jkfifi fM is called the provisional expected frequency of the kth categorical response 
of item j at the fth quadrature point. 

The item category parameter, b^, is contained in both P^Cg) and P^+iCfi) as shown in 
[18]. The first derivative of Equation 39 with respect to is given by 

Therefore, the element of the gradient vector is numerically computed as 

t = v r v r r j* f i f a-- f « - • r j>*+ i ' f i f 2~ f « i dp* jk {K) f521 
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The elements of the information matrix are given by 



V =T TYN f 1 dP *™ dP *™ [53] 



*2 F ± 



- E ~E E ^[piv) ^ Sa- i £%^ 2 t54] 



v = -v f fj? i e^(20 apj^xcz) [55] 



and, when Jk-k'l ^ 2, 



***** = 0 t 5 «] 



and finally 



jf ^- f " L P jk (X) da jtt aa, a J §5^ 

[57] 

The algorithm presented above was implemented in POLYFACT (Muraki, 1993). The 
POLYFACT is a hybrid computer program of PARSCALE (Muraki & Bock, 1993) and 
TESTFACT (Wilson, Wood, & Gibbons, 1984). The program computes the factor loadings 
by the principal factor analysis based on the product moment correlation matrix, treating item 
responses as a continuous variable. Because the factors of the principal factor analysis are 
orthogonal, their loadings are suitable for the full-information solution after conversion to 
slopes. Slope estimates based on the full-information method are then converted again into 
factor loadings. The resulting full-information factor loadings are then rotated orthogonally 
to the varimax criterion (Kaiser, 1958) and, with the varimax solution as target, rotated 
obliquely by the promax method (Hendrickson & White, 1964). 



Simulation Study 

The polytomous item responses of six items and 5000 respondents were generated by the 
RESGEN (Muraki, 1992b) computer program. All six items have three categorical 
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responses. . The values of the original parameters are presented in Table 1 . The values of 
a=1.2 and a=0.0 correspond to a=0.768 and a=0.0, respect/ely. The communality is the 
same for all six items (hj 2 =0.590, j=l,2,...,6). 



******************************** 
In.sert Table 1 here 

******************************************** 



The three-dimensional solutions were obtained. Five quadrature points for each 
dimension were used for numerical integration. Therefore, 5 3 total points were used for 
integration. The precision for convergence was set 0.001. Nineteen EM cycles were needed 
to reach convergence. The estimated slope parameters were converted to factor loadings, 
and the communality for each item was computed. The factor loadings were then rotated 
with a varimax criteria and they are presented in Table 2. The slope parameters were 
recovered from the varimax factor loading, and they are shown in Table 3. The item- 
category parameters are also presented in Table 3. Reckase's multidimensional 
discrimination parameter (MDP) and direction cosines were computed based on the varimax 
factor loadings and are shown in Table 4. 



******************************************** 

Insert Tables 2, 3, and 4 here 
******************************************** 



All of the slope parameters were underestimated, and consequently the varimax factor 
loadings are lower than the original ones. The item-category parameters are also 
underestimated. The underestimation of these parameters may be eased by increasing the 
number of quadrature points. We are currently investigating this possibility. Nevertheless, 
the factor structure of the original simulated data is recovered in die estimated parameters. 
The process of conversion from the slope parameters into factor loadings and the rotation of 
the factor loadings is an efficient way to study the results of the analysis. We can then 
recover the slope parameters and compute the multidimensional discrimination parameters 
and the direction cosines. 

For further research, we are planning the analysis of real data sets of polytomous item 
responses. The test of chi-square fit is essential to investigation of the appropriateness of the 
multidimensional item response models. The interpretation of the multidimensional 
parameters in terms of classical factor analysis is also needed. Since increasing the 
dimensionality requires an exponential increment in computational time, we need to 
investigate adjusting the number of quadrature points to economize the estimation process. 
In this research, we attempted to estimate the parameters of the multidimensional graded 
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response model. We succeeded in this basic task. This is the first step in expanding the 
application of the factor analysis to qualitative data. 
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Table 1 



Original Model Parameters 



Item 


ai 


Slope Parameter 

a 2 


a 3 


Item-Category Parameter 
bi b 2 


1 


1.2 


0.0 


0.0 


-1.8 


0.8 


2 


1.2 


0.0 


0.0 


-0.8 


1.8 


3 


0.0 


1.2 


0.0 


-1.8 


0.8 


4 


0.0 


1.2 


0.0 


-0.8 


1.8 


5 


0.0 


0.0 


1.2 


-1.8 


0.8 


6 


0.0 


0.0 


1.2 


-0.8 


1.8 



21 



19 



Table 2 



Varimax Factor Loadings and Communality 





Factor Loadings 




Pnm m 1 1 na 1 i tv 
v^miiiiiui la 1 11 y 


Item 


«i 


a 2 


a 3 


h 2 


h 


1 


0.709 


0.002 


-0.007 


0.503 


0.709 


2 


0.705 


-€.002 


-0.014 


0.497 


0.705 


3 


0.006 


0.699 


0.021 


0.489 


0.699 


4 


-0.006 


0.698 


0.003 


0.487 


0.698 


5 


0.012 


-0.017 


-0.723 


0.523 


0.723 


6 


0.010 


-0.009 


-0.729 


0.532 


0.729 




Table 3 



Estimated Model Parameters 





Slope Parameter 


Item-Category Parameter 


Item 


ai 


a 2 


a 3 


b, 




1 


1.005 


0.003 


-0.010 


-1.204 


0.492 


2 


0.994 


-0.003 


-0.020 


-0.546 


1.211 


3 


0.008 


0.978 


0.029 


-1.121 


0.505 


4 


-0.008 


0.975 


0.004 


-0.522 


1.161 


5 


0.017 


-0.025 


-1.047 


-1.149 


0.549 


6 


0.015 


-0.013 


-1.065 


-0.507 


1.191 



23 



21 



Table 4 



Reckase's Multidimensional Parameters 





MDP 


Direction Cosines 


Item 


1] 


X, 


x 2 


x 3 


1 


1.005 


1.000 


0.003 


-0.010 


2 


0.994 


1.000 


-0.003 


-0.020 


3 


0.978 


0.009 


1.000 


0.030 


4 


0.975 


-0.009 


1.000 


0.004 


5 


1.047 


0.017 


-0.024 


-1.000 


6 


1.065 


0.014 


-0.012 


-1.000 
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Figure 3. Multidimensional Discrimination for a Two-Dimensional Space. 
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